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The Early Development Instrument (EDI) was designed by McMaster University to measure the 
outcomes of children’s early years as they influence their readiness to learn at school. It is a 
group measure of “‘at risk” that assesses five different domains of early development: 
e Physical Health and Well Being 
Social Competence 
Emotional Maturity 
Language and Cognitive Development 
Communication Skills and General Knowledge 


The EDI was piloted in several Canadian cities in recent years through two national initiatives — 
the “Understanding the Early Years” (UEY) project and the National Longitudinal Survey of 
Children and Youth (NLSCY). Building on these initiatives, Alberta Learning piloted the EDI as 
a possible measure of readiness to learn at school in two school iueccnenes in 1999/00 and in 
five jurisdictions in 2001/02. 


This study evaluated Alberta Learning’s pilot project It examined the efficacy and utility of 
the EDI for teachers’ use in classroom programming and ea somone and 
addressed three specific research questions: 
1. Was the support and organization provided during the pilot project adequate for use of the EDI] in 
schools? 
2. Was the data generated from the EDI and provided to schools useful in predicting performance in 
the early grades in school, as well as for teacher program planning? 
3. Is the EDI appropriate as a province wide measure of readiness to learn at school? 


Semi-structured telephone interviews were conducted with samples of teachers and administrators from 
the five jurisdictions that piloted the EDI in 2001/02, and the five project coordinators. Several factors 
impacted the results of the pilots and the evaluation: 


e Late receipt of the EDI results from McMaster University at both the a and school level. 

e Difficulties in contacting targeted interviewees. 

e The active parental consent requirement, which limited participation in the pilot and affected the 
representativeness of the results (consequently, they must be interpreted with caution). 


Most participants found the EDI easy to use, and viewed the organization and support provided 
for its administratiofi as ‘very adequate’. The orientation sessions were viewed very positively, 
and release time to administer the instrument was considered. extremely helpful. Three 
suggestions for improvement were received: . 

e Distribute the EDI instrument in advance of the orientation sessions so Eeeieee can 

become familiar with it. 
e Include more case studies for year 2 teachers who were also involved i in year 1. 
e Administer the EDI earlier in the school year (e.g., March or April). 


The majority of those interviewed were positive about the concept of assessing cohort 
performance, particularly regarding school-to-school or system-to- system comparisons. At the 
same time the majority of respondents did not find the EDI particularly useful for classroom 
programming and predicting cohort performance in their particular classroom, as they were 
already familiar with their students. Some teachers commented that the EDI made them more 
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aware of children’s physical well-being. School administrators were often unaware of the 
piloting of the EDI in their schools or of the data provided, 


‘About half the teachers interviewed felt the EDI or similar assessrrient should be eanlemnented 
province wide under certain conditions. On the other hand, the majority of coordinators and » 
administrators indicated this would not be appropriate, and the few who indicated-a tentative 
“yes” also attached some conditions. Responses of “it depends on the purpose” also were 
voiced. . : 


This evaluation also included a literature review on assessment of the psychomettic properties of 
the EDI instrument. While the EDI has face validity, there are no definitive studies of its 
soundness. Some research suggests the EDI has reasonable inter-rater reliability; however, 
concurrent and predictive validity have not been established. No published studies on the 
reliability and validity of the EDI were found. 


RECOMMENDATIONS 

1. Further evaluation should be done to establish the psychometric properties of the EDI before 
using it for predicting children’s readiness for school, or for school or classroom 
programming. 

2. ‘The EDI could be improved - 
® including some auditory analysis skills assessment and other metacognitive skills 

assessment (e.g., cognitive and behavioral monitoring). 
e doing a factor analysis to reduce overlap of items within and among the five domains, 
thereby shortening the number of items to be administered. 
The strength of the EDI is that it looks at the child as a whole - the physical, social, 
behavioral and emotional domains as well as the cognitive. This should be retained if the 
instrument is refined.. 
3. Ifan improved EDI is developed, the following learnings from the pilot project should be 
considered to improve implementation: 

e Stakeholders at all levels should be involved at all stages of implementation. For 
example, administrators should be involved in in-service activities and discussions so 
they can better support teachers and ensure utilization of the data more broadly at the 
school and in the community. 

e In-service'activities should address how the data should be utilized in making 
educational, social, financial, community and/or health related decisions. 

¢ To maximize its utility, EDI data should be collected in February or March, and the 
results provided to schools in May, of the same at and no later than the beginning of 
the next school year. 

e Data collection should be representative of the c target Population to ensure the validity 
and utility of the results. 

e Since data collected through the EDI reflect children’ s total environment, the educational 
community should involve social, health and recreational agencies in interpreting results 
and coordinating follow-up activities (perhaps through the Alberta Children and Youth 
Initiative and its membet ministries) for maximum leveraging of resources to address 
developmental needs of young children, especially those ‘ ‘at risk”. 
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Predicting children’s competencies in the early school years has a long history in the educational 
arena. Important educational decisions for children such as labeling, tracking and retention are 

' often made on the basis of their performance on readiness tests (Meisels 1999). The goal of 
having children start school ready to learn is indeed an important one, but one which has had a 
very problematic history. The Federal Government’s initiative “Understanding the Early Years” 
is a national research initiative which attempts to “....strengthen the research capacity of 
communities to make informed decisions about the best policies and most appropriate programs 
to serve families with young children”. This project involved among other things data on the 
early development of children aged 5 to 6 as reported by their teachers on the Early Development 
Instrument (EDI) as well as the National Longitudinal Survey of Children and Youth 
Assessments (NLSCY) involving vocabulary skills, developmental level Who am I? test, number 
knowledge assessment and behavior outcomes (structured interview with parents or other care 
givers). The Early Development Instrument (EDI) is used in an attempt to determine the . 
numbers of children considered “at risk” in five domains of development including, Physical 
Health and Well Being, Social Competence, Emotional Maturity, Language and Cognitive 
Development and Communication Skills and General Knowledge. This is a group measure of 


“at risk” and is interpreted with respect to groups and not individuals within. 


The study reported here attempted to evaluate this particular instrument with regard to teachers’ 
perceptions of efficacy and usefulness of this instrument to provide information helpful for 
making instructional decisions and programming within the classroom. The following sections 
provide first a brief overview of the concept of readiness and some issues involved followed by a 
brief discussion of the assessment of readiness, including the Early Development Instrument. 
This is followed by a review of some of the available information regarding the use of the EDI in 
the national context. The final section presents evaluation data from a pilot study conducted in 
five Alberta school jurisdictions. The last section provides some recommendations based on 


study results as well as the research literature. 
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READINESS 


The Canadian Document Zero to Six: The basis for school readiness, written by Gillian Doherty, (1997) 
states that “school readiness refers to the student’s ability to meet the demands of school, such as sitting 
quietly, and to assimilate the curriculum content at the time of entry into the formal school system .... 
Being ready for school entry gives the opportunity to benefit from all that school has to offer, both 
academically and socially” (p. 1-2). Although true, this definition in and of itself implies that the 
characteristics lie within the child and possibly that we must wait for the child to develop these 
characteristics. A more recent elaboration of the traditional definition of school readiness emphasizes 


the importance of the school being “ready for the child” (p. 13). 


The components of school readiness may be thought of as a child’s level of physiological maturation, 
innate abilities, temperaments and early experiences. Atterition in kindergarten has been found to a 
good predictor of a child’s later reading skills (Horn & Packard, 1985). Also included in a child’s 
readiness are social skills and motivational factors. Inadequate social skills have been-associated with 
early departure from school (Amble, 1967; Barclay, 1966; Gisniand & Holmlund, 1958; Janes, 
Hesselbrock, Myers & Penniman, 1979), and in some cases delinquency in early adolescence. Some 
theorists and researchers would suggest that these social skills difficulties can be traced back to 
kindergarten entry (Ensminger, Kellam, & Rubin, 1983; Tremblay, Masse, Perron, Leblanc, 
Schwartzman & Ledingham, 1992). Some researchers have also suggested that children who lack the 
necessary social skills tend to resort to aggression and bullying (Coie & Kupersmidt, 1983; Dodge, 
1983; Ladd & Price, 1987). = 


According to the Zero to Six: The basis for school readiness Canadian Document, school readiness 


involves the following five components within the child: 


e Physical well-being and appropriate motor development; 

e. Emotional health and a positive approach to new experiences; 
e Age-appropriate social knowledge and competence; 

_¢ Age-appropriate language skills; and 

e Age-appropriate general knowledge and cognitive skills. 


The order of these components is not arbitrary. It highlights the growing importance of components 


other than academic skills as contributing to school readiness. 


Physical well-being and appropriate motor development not only includes the ability to fight off disease 
or serious physical handicapping conditions, but also encompasses the child’s ability to have adequate 
levels of energy to concentrate in school and resist common infections.. Other areas include both gross 
and fine motor development. Adequate gross motor development has been related to self esteem issues. 


-Fine motor skills are required to complete the curriculum. 


Emotional health and a positive approach to new experiences suggest that the child should be able to: 


e ' Defer immediate gratification; 
e Persist in repetitive exercises; and 


e Cope with momentary failures. 


For example, a child who would be considered as exhibiting a positive approach to new experiences 
would be the one who falls between the child described as fearful and reluctant to engage in new 


experiences on the one hand, and the one described as impulsive. 


Age-appropriate social knowledge and competence is required for the child to engage in acceptable 
behavior in a public place. There is a need for children in school to accept adult authority as well as to 
work cooperatively with their peers. Research has found that even when children have adequate 

. cognitive and language abilities if they do not have social knowledge and competence they tend to 
perform lower than expected on both reading and math, as well as having impaired social development. 
It has also been found that behaviors such as aggression and lack of cooperation can persist overtime 
and across settings leading to peer rejection (Coie & Kupersmidt, 1983; Dodge, 1983; Dodge, Coie & 
Brackke, 1982; Ladd & Price, 1987; Putallaz & Gottman, 1981). This again has been known to lead to 


early departures from school. 


Age-appropriate language skills include the ability to: | , 


e understand adult and other children’s verbal communication; and 
e verbally communicate experiences, ideas, wishes, and feelings in a way that can be understood by 


others. 


Research certainly supports at least at the moderate level that language, general knowledge, and 
cognitive skills at Kindergarten or grade one entry are predictive of future academic performance (Hess, © 
Holloway, Dickenson & Price, 1984; Reynolds, 1989; Tizard, Blatchford, Burke, Farqhuar & Plewis, 
1988). In addition, oral language ability in kindergarten is said to account for thirty to forty percent of a 
child’s later reading ability (Adams, 1990; Biemiller & Doxey, 1993; Reynolds, Mavrogenes, Bezruczko 
& Hagemann, 1996; Yopp, 1988). | 


Age-appropriate general knowledge and cognitive skills as mentioned above have been found to be © 
moderately good predictors of academic success in later grades. These general knowledge skills include 
such things as knowledge of board games, and understanding of sequencing. Cognitive skills include 
children’s ability to perceive, organize and analyze information provided in their environments. Also 


included are short and long term memory, as well as the ability to identify similarities and differences. 


Research would suggest that children’s development and school readiness, is an interaction of genetic 
endowment, physiological maturation, and the active engagement with the environment. As well, each 
developmental stage is dependent on the preceding stage. It has also been found that some:knowledge 
and skills required for adult competencies are rooted in the first six years of life. There is also research 
to support the idea of ‘critical times’ for development. These ‘critical periods’ are specific 
developmental periods when the child is biologically primed to develop more advanced neural structures 
and/or skills provided that appropriate environmental stimulation from parents, siblings, child care 
providers, or peers is provided. That is, interactions between the child and his/her environment are key 


_ determinants of.the child’s school readiness... 


However, although some physical limitations have been found to be irreversible much of what is taught 
and acquired in the first six years can be remediated successfully during this time period. Children who 


are late talkers through speech language intervention/therapy can and do develop into strong academic 


child is not ready to go to school, but it would identify here the feeds are and what is fequited 0 alist 
them in becoming ready to benefit from the curriculum. In fact, it has been said that waiting for a child 
to develop skills may be the worst approach as it is during this young age that the child is most likely to 
develop the needed skills. 


Of particular importance to this report is the tremendous variability that 5-year-old First Nations English 
speaking children can display. Research has suggested that the language skills, as well as the social and 
fine motor skills of these children may differ by a year or more (Biemiller & Doxey, 1993). In other 
words, schools need to be responsive to the wide ranges of children’s developmental levels and 


backgrounds at kindergarten. 


- In the end, Doherty identifies several areas of continued research, including family factors, parental 


workforce participation, parental stress levels, community resources and family-friendly workplace 
policies and practices as impacting school readiness, and how these factors support-children’s school 


readiness. 


There are a number of other articles that might be useful in exploring this concept of school readiness. 
An article, Readiness: Children and schools, written by L. Katz (1991) provides a summary of the 
school readiness concept. There were two views of readiness proposed, namely, school readiness due to 
maturational processes or due to environmental experiences. Katz highlights the role of early 
experiences, children’s physical and emotional well-being as well as cognitive relationships as 
impacting readiness: However, as stated in Doherty’s article, readiness goes beyond cognitive abilities 


to include a social component, such as getting along with peers and accepting adult’s authority. 


For a comprehensive review of the four main models/theories of school readiness, refer to G. Gredler’s 
book titled School Readiness: Assessment and educational issues, (1992). Gredler also examines five 
issues related to early school readiness; the rationale for early screening, the instability of children’s 
behavior, the use of inappropriate measures, the difficulties in defining who is “at risk’’, and the 


problems in accurate prediction. 
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Kagan’s (1992) seminal paper on school readiness provides operational definitions for the concept of 
learning readiness since the early 1980s. Doherty’s paper referenced her work extensively. The main 
thrust of-the paper was to focus on the two competing conceptions of “readiness to learn” as more 
developmentally based, and “readiness for school” as focusing on specific cognitive and linguistic skills. 


These two terms are often used interchangeably. 
ASSESSMENT of READINESS 


The previous Canadian Document regarding readiness provides general information, as well as research 
to back up the statements made. However, it does not address the specifics of assessing readiness, or 
identify the potential pitfalls that can occur. Shore (1997) asks some ethical, theoretical and practical 


questions regarding the assessment of readiness. 


- Can children’s school readiness be assessed without doing them harm? 
Can readiness assessment avoid labeling or stigmatizing children? 


. Will preschool programs become distorted if they ‘teach to the test’? 


ere he Es 


Is it possible for readiness testing to recognize the unique character of early development and 
learning? - . 
5. If large numbers of children are not ready for school, will this be viewed as a problem in the child or 


... Within the community? - 


It is imperative that those responsible for deciding on readiness testing be well versed with the 
implications of testing and the possible turns it can take. Often readiness tests are an earlier version of 
skills that are assessed by achievement tests at more advanced levels later. It is the child’s relative 
preparedness to take advantage of a specific program or curriculum by describing the child’s current 
level of skill achievement or pre-academic preparedness. However, as mentioned above, readiness is a 
bi-directional concept that looks at current skills, knowledge and abilities along with the child’s early 
experiences at home. How does one weigh the importance of both these aspects of learning? Are there 
valid and reliable assessments that measure both areas and is there information to state which of these 


two main concepts is the better predictor of later school performance? 
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grade: developmental screening measures and readiness measures. These two naies lave been at times 
used interchangeably. According to Meisels (1994), developmental screening measures are meant to 
assess the developmental level of a child’s potential to acquire skills. On the other hand, readiness tests 
tend to assess cognitive and linguistic skills assumed to be related to school learning tasks that are, in 
turn, assumed to be predictive of future school achievement. An analysis of three readiness tests, (i.e., 
Brigance, DIAL-R, and the Gesell) and two screening measures (i.e., Early Screening Inventory and the 
Denver II) showed considerable overlap as to the actual skills assessed (see Glascoe, 1995 for an 
extended analysis). Furthermore, regardless whether the instrument is a developmental screening _ 
measure or a readiness test, there appears to be little difference as to the kinds of decisions and goals 


reached by school personnel. . 


._ Aligning Assessment Tools to Goals 


Salvia and Yesseldyke (2001) in their book Assessment devoted one chapter to the tests used with 
preschoolers for the purpose of screening. The focus generally was on the identification of children who 
would profit from early intervention. The major measures in the area are reviewed and evaluated with 
respect to reliability and validity. The majority of the measures were found to be reliable, but the 
validity was often questionable. One of the greatest problems with readiness tests is their lack of 
validity (Meisels; 1998) which creates a danger of misclassification. Also identified in the assessment 
of preschool children was the misuse of instruments. Specifically using a test developed for one purpose 
but used to measure for another purpose. For example, many screening devices such as the Brigance K 
and 1 screening form are used for program planning rather than using the screening form to make an 
appropriate referral for more in-depth assessments, if warranted. For a thorough analysis of the. 
reliability and validity properties of the Brigance K and 1 screen, refer to Mantzicopoulos’ study (1999) 
of a sample of Head Start children. It is disconcerting that many districts often make placement or 
programming decisions based almost completely on the results of one assessment (possibly lacking 
reliability or validity), whether it is a readiness or screening device, rather than using the assessment as a 
referral for possible further assessment and/or monitoring. Namely, labeling, tracking, denying access 


to kindergarten, and retention are often made on the basis of readiness performance. 
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Readiness measures are described as a special form of preschool assessment, administered for the 
purposes of predicting who is not ready for formal school entry and who will profit from remedial or 
compensatory interventions. Thatis, readiness for school suggests that a specific set of cognitive, 
linguistic, social and motor skills must be attained to indicate readiness. Readiness tests describe the 
child’s entry characteristics (Meisels, 1987). They are skills, or criterion-referenced and are designed to 
help teachers plan how they will teach the curriculum. On the other hand, screening tests are norm- 
referenced and assess a child’s potential likelihood of acquiring skills, as well as degree of risk. Asa 


result, three major dilemmas are inherent in the testing of young children: 


e Tests are administered for the: purpose of predicting later performance, but at these young ages, 
performance is so highly variable that predictions are very difficult. 
e Jtis dangerous to use preschool screening measures as indices of current standing. 
_e Provision of services is dependent on labeling children but labeling may set up expectations for 


limited pupil performance. 


The variance in young children’s behavior often causes problems in attaining reliable assessments. 
Making predictions from these same assessments is equally or even more problematic. According to _ 
Gredler (1997), “for early identification to make sense, there must be sufficient consistency over time 
between a child’s early functioning and what is educationally significant for the child in later years” (p. 
100). . 


iC Children’s Screening Devices and Predictive Validity 
A meta analysis undertaken by La Paro and Pianta (2000) focused on 70 longitudinal studies, across 
time from 1985-1998. A gap in the research existed as to quantitative estimates of effect sizes that 
might exist for relations between preschool or kindergarten academic/cognitive and social/behavioral 
assessments and early school outcomes. This current meta-analysis attempted to address the question of 
-the stability of individual differences across time from preschool or kindergarten to grade land/or grade 
_2. La Paro and Pianta found that academic/cognitive assessments showed moderate effect sizes for 


longitudinal studies predicting school success. However, the social/behavioral predictors of early school 
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variables) were not taken into consideration. 


It is important to note that the higher correlations reflected studies in which children were assessed with 
either the same or very similar measures at.Time 1 and Time 2, and the lower correlations generally 
reflected studies in which children were assessed with a battery of measures at Time 1 and Time 2. 
They concluded that “instability or change may be the rule rather than the exception during this period” 
(p. 476). 


The meta-analysis provided support for assertions from previous qualitative and anecdotal reports that 
defining and assessing “‘school readiness” is problematic if we only focus on children’s skill and 
abilities, while ignoring how these interact with the environment. Early childhood equcnine continue to 
emphasize the importance of social processes in school success. Certainly, Doherty’s (1997) report 
placed physical and emotional components and social processes in a hierarchy of importance as the top 


three, followed by cognitive and language abilities. 


The Problem of a High Rate of False Positives 

Because the false positive rate of a large number of screening tests is quite high (see Gredler, 1992), 
users of such measures must use extreme care in the interpretation of the results. As previously stated, 
particularly problematic is predictive validity. To establish predictive validity two criteria are needed. 
The number of children identified as “at risk” and “not at risk” by the particular test and some measure 
of performance indicating those children who performed adequately in school and those who did not. 
When correct predictions are combined, a “hit rate” is obtained. For example, one of the most reliable 
readiness tests, the Metropolitan Readiness Test, has a 30% error rate when used for placement purposes 
(Shepard & Smith, 1988). In fact, the incidence rate of false positives has been underestimated. As a 
result, undue stress for both the parents and child may result in increased anxiety. In the United States, 


false positive scores on developmental screening or readiness tests have been used to deny entry to 
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kindergarten or grade one because the child is deemed not to “be ready”. Although our understanding 
from a review of Canadian literature suggests that these assessments are not being used in this way, it 


still begs the question of how they are being used to make decisions about children’s school readiness. 


Gredler (1997) discusses a promising method for calculating the predictive validity of a screening 
measure and thereby decreasing the number of false positives. Computation is derived from the four 
possible outcomes of the administration of a screening measure: number of children identified as at risk 
who later perform poorly, number identified as not at risk who later perform poorly, number identified 
as at risk who later perform adequately, and number identified as not at risk whose later achievement is 
adequate. Gredler developed the “index of sensitivity” and the “index of specificity” resulting in four 
predictive indices. Although all four indices are important in determining the effectiveness of screening 
measures, the positive predictive value is the index that school personnel would probably use in making 
decisions about children “at risk”. In using the four predictive indices to assess the predictive validity of 
the Brigance K-1 Screen, Denver II, the DIAL-R, Phonological Measures, and the Early Screening 
_Inventory, Gredler concluded that the level of inaccuracy in accurately identifying children was: 
problematic. Of these five measures, the Brigance K-1 Screen and the DIAL-R had better positive 
predictive values than the other three measures. A major problem is that a correlation coefficient 
between a group’s score on a preschool screening instrument and a later achievement measure is usually 
_ provided in the literature as evidence of the test’s effectiveness. Such data, although useful, only 
provides information on the similarity of the group’s performance on both tests: In other words, a 
correlation coefficient provides no information as to the specific identification of “at risk” and “not at- 
risk” children and the relationships between such status and the predicted outcome of a group (Satz & 
Fletcher, 1988), | | 


An analysis of twelve screening measures by Gredler (1992) indicated an average sensitivity.index of 
.77, an average specificity ratio of .81, anda positive predictive value of .55. This predictive value of 
..55 means that 45 out of every 100 children designated as being “at risk” were able, contrary to 
predictions, to demonstrate adequate reading mastery in subsequent school years. Thus, as Cadman, 
Chamber, Feldman, and Sackett (1984) have stated, “a test with a low predictive value is unlikely to be 
either efficient or useful .. .” (p. 1583). Another analysis of 8 screening devices by Carran and Scott 


(1992) yielded similar findings with regard to predictive validity. 
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The revids of literature would suggest that when gonudeang eadines testing, the error rate in the use 
of predictive measures at this age level is more problematic than at first thought. Some researchers have 
suggested that school readiness.is invalid (see Shepard, 1997 for a review of technical and policy aspect 
of readiness testing in local U. S.school districts). 


The Invalidity of School Readiness Testing 


The following comments are taken from Shepard’s Review (1997) of local U. S. school districts and 
echoes the suggestions made by several national reports. Namely, that there is no validity to the concept 
of readiness as currently constructed. Instead, Shepard proposes that “all children are ready to learn 
something at every age” (p. 91): Instead of readiness as the main focus, these national position 
statements have focused on two main themes: “Developmentally appropriate curriculum” (Bredekamp, 
1987) and making schools ready for children. Most importantly, developmentally appropriate _ 
curriculum addresses both typical developmental levels of children in classrooms as well the large range 
(variability) of abilities and skills of this age group. Therefore, teachers need to be prepared to meet the 
heterogeneous skills and abilities of children in early grades. Shepard’s main argument centers on the 
fact that readiness tests lack the technical rigor to make accurate predictions or placements. Although at 
first, it appears that Shepard is ‘throwing out the baby with the bath water’, in the end she proposed that 
further research would be warranted to establish whether there is any evidence of earlier or. more 
accurate identifications of students with special needs where all students must be tested. This is 
certainly the thrust of the Who Am IJ? and Early Development Instrument (EDI) ) discussed in the 


following:section.* 


DISCUSSION OF THE EDI and WHO AM I? MEASURES 
Who Am I? Instrument 


In Canada, Who Am I? has been used in a number of communities participating in the 
national initiative on Understanding the Early Years (UEY), as well as in the fourth set of 
the Canadian National Longitudinal Survey of Children and Youth (NLSCY). The Who 
Am J? instrument was developed for use in a research study undertaken by the Australian 
Council for Educational Research (ACER) in 1998. The Who Am I? measure was 
developed to assess children’s level of development at preschool and entry to school 
level, as well as their readiness for formal schooling. It is based on early copying and 
writing skills, and is designed to identify the broad stages of development that underlie 
children’s readiness for school learning. The Who Am I? has now been piloted in 
Australia, Canada, India, Sweden, and Hong Kong. 


The purpose of the Who Am I? instrument in the Canadian studies was to provide one of 
three direct measures of children’s early learning and development. The information can 
then be used to examine the association between early development levels and learning 
and various community and family factors that are considered to be associated with early 
development that in turn may impact children’s school readiness.and later success in 
school and life. The Who Am I? instrument can be used to identify children “at risk” for 
school difficulties. A brief review of the technical data suggests that it has good | 
reliability, indicating a high level of internal consistency. It also has good test-retest 
reliability and inter-rater reliability. With regards to validity, it appears to have 
reasonable construct validity. In addition, reasonable attempts have been made to 
provide for concurrent validity using scores on the Literacy Baseline test and the scores 
on “I Can Do Maths”. It is planned that Canadian norms will be developed using data 
from the NLSCY studies. What is missing from this measure is the importance that has 
been placed on a child’s physical, emotional and social well-being. Another instrument 
which attempts to incorporate these domains that has been used in the UEY studies is the 


Early Development Instrument (EDI). 
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The Early Development Tngirument 


The Early Development Instrument (EDI) is designed “to measure the outcomes of 
children’s early years as they influence their readiness to learn at school 

(Description of EDI used in the Introduction to Alberta 2001/2002 pilot study), 
conceptualized as a child’s ability to meet the task demands of school, such as being 
cooperative, sitting quietly and listening to the teacher, and to benefit from the 
educational activities that are offered by the school” (Description of EDI used in the 
Introduction to Alberta 2001/2002 pilot study, Doherty 1997; Janus & Offord, 2000). 
The EDI measures how ready children are to begin learning at school by asking questions 
in five different domains of early development: physical health and well being, social 
competence, emotional maturity, language and cognitive development, and 
communication skills and general knowledge. The scales are completed by Kindergarten 
teachers. Each domain is scored on a 0-10 scale, where 0 is the worst, and 10 is the best 
score. The results for each domain are then divided into percentiles (based on the entire 
sample of students) to facilitate interpretation. Children with scores in the lowest 10% of 
the data (below the 10" percentile) in one or more domains were considered as having 
serious problems. The results are interpreted for groups of children and no individual 
diagnostic information is provided. As a result, the percentage of children in the 
population who score in the lowest 10" percentile in one or more domains could be 
interpreted as an indication of how many children are “not ready to learn at school” or 
have “increased needs” in comparison with the rest of the population. Descriptors of 
children who scored in the lowest 10% on each of the five domains are provided by the 


EDI. 


The results are often applied at the macro (province, community, school board etc) and 
micro (neighborhood to neighborhood, school to school, or class to class etc) levels. The 
test developers have proposed that considering the results at the micro level is ‘like taking 
a magnifying glass to the results at the macro level’. At this level the EDI are analyzed 
school by school, and neighborhood by neighborhood. The micro level shows how 
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children in a particular neighborhood are doing. At the school level it will indicate areas 
of strengths that have to be cultivated, and will point to areas of difficulties that students . 
may have that need to be addressed. It has been designed to provide information for 
groups of children to: 1) report on populations of children in different communities, 2) 
monitor populations of children over time, and 3) predict how children will do in 
elementary school. A review of literature with regard to the psychometric properties of 


the EDI was undertaken. There were no published statistics regarding this found. 


In a personal phone communication (March 24, 2003) with one of the EDI developers, 
Magdalena Janus, it was reported by Dr. Janus that no formal published studies have been 
conducted with the EDI with respect to validity including predictive and construct validity 
but these are being considered and some work in the direction is currently being conducted. 
It would appear through examination that this instrument has good face validity; however, no 
concurrent or predictive validity testing has been reported. One unpublished study re: the 
inter-rater reliability with regard to the questionnaire was sent via the internet by Dr. Janus. 
This reported reasonably good inter-rater reliability; however, test-retest or internal 


consistency evaluation was not considered and limited data regarding validity was presented. 


The EDI appears'to be a hybrid instrument in that it is described as a readiness test that is criterion 
referenced as well as being a norm-referenced screening tool for groups of children. It is therefore, 
somewhat difficult to compare it to most of the available screening and readiness tools addressed in this 
report. However, the issues covered still apply to the EDI as well. Namely, to what purpose does one 
have in using the EDI? The EDI has to have sound psychometric properties at the group level as well. 
If one decides to use this as an instrument to make diagnostic decisions of individual students, its 
psychometric properties have to be even more rigorous than for groups of children. With regards to 
reliability, there appears to be reasonably good inter-rater reliability. Test-retest or internal consistency 
evaluation has not been addressed. With regards to validity, predictive, construct and content validity 
have to be addressed as well. It appears that its construct and content validity is drawn from Doherty’s 
report (1997) on the five components of school readiness which in turn draws heavily from Kagan’s 
(1992) seminal work on defining school readiness. From our review of the available material, this is no! 


explicitly stated by the EDI developers. 
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Face Validity of the EDI 


Certainly the items on the EDI represent a large extensive number of items that are found in various 
readiness instruments. There appears to be some overlapping of the areas which are something that is 
often observed in these instruments. It would be helpful to have some construct validity assessment. 
Perhaps a factor analysis of these items may help decrease the number of items in each area which may 


in turn help to decrease the time to administer, as well as identify areas of strengths and weaknesses. 
A few recommendations to improve on this device would be to: 


e Take an assessment that has been successful but missing various items, such as perhaps auditory 
analysis skills and metacognitive strategies that have been shown to have predictive validity with 
regard to later reading success, and adding these to the measure. The next step would be to norm 
the results on the pre-school population. 

e Establish concurrent validity on the group of children in the lowest 10% on the EDI, using other 
‘instruments such as the Brigance, or Metropolitan Readiness Test to better establish if the lowest 
10% of the children are really at-risk, or is another cut-off more valid in a particular group of 
children. | 

@ Develop a new measure with sound psychometric properties that is standardized and normed on 


a randomly selected, representative sample. 


The next section briefly reviews the use of the EDI across communities in Canada. 
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NATIONAL EDI REVIEW 


The EDI has been implemented in a number of sites across Canada through the Understanding 
the Early Years (UEY) national research initiative of HRDC. It has been implemented along 
with the National Longitudinal Survey of Children and Youth (NLSCY) Community Survey 
which gathers data directly from parents and children. The EDI provided, among other ihings, 
information regarding the experiences of children and families as well as child outcomes 
regarding cognitive skills, prosocial, and behavioral outcomes. Behavior outcomes were 
assessed through in depth telephone interviews with parents or other persons most 
knowledgeable about the child. Cognitive and literacy skills were evaluated through the 
Peabody Picture Vocabulary Test, the “Who Am I?” test of early literacy and number knowledge 


assessment. 


The sites across Canada included among others, Abbottsford, British Columbia; Prince Albert, 
Saskatchewan; Winnipeg, Manitoba; New Brunswick, Prince Edward Island, Newfoundland, 
Nova Scotia, Ontario and Nunavut. A phone request was made to Dr. Magdalena Janus to obtain 
the addresses, phone numbers or e-mail addresses of contact persons at sites currently involved. 
This was denied due to concerns regarding confidentiality. However, a number of provincial 
sites were contacted through e-mail addresses available through the UEY web site. Some 
documentation was available for Prince Albert, Saskatchewan; Abbotsford, British Columbia; 


Winnipeg and Saskatoon. 


The Prince Albert project administered the EDI to a sample of 433 kindergarten children and 339 
of these protocols were analyzed. Ninety-four students were not included in the final analysis 
due to missing scores. The results indicated that the areas of greatest concern were Language 
and Cognitive Development, and Physical Health and Well Being, where greater percentages 
with a low score were evident as compared to national sample scores. The sample scored above 


the comparative sample with regard to Social Knowledge and Communication Skills. 


The EDI was utilized in this project in conjunction with direct assessments of the child and 


behavior outcomes (through parent interviews). The purposes of this project were to assess how 


As well, the project attempted to determine how important certain family and community factors 
_ were in affecting child’s development. The data is being used to provide information as to how 
the community can act over the next few years. The information would appear to not be used 


directly for educational programming or individual children’s needs. 


The EDI was also implemented in Winnipeg (School Division No. 1) in 2001. Outcomes for — 
children were evaluated utilizing the same measures as those for the Prince Albert project. The 
sample consisted of 511 Kindergarten children. As was the case with the Prince Albert study, a 
greater number of children performed below the 10" percentile in both Physical Health and Well 
Being, and Language and Cognitive Development. The data from this project was not apparently 
utilized to make educational programming decisions, but rather in an attempt to mobilize needed 


community resources. 


In a study in 2001-2002, a random sample of 425 kindergarten students were selected in 
Saskatoon and were administered the same individual assessments and teacher EDI 
questionnaires as in the above two projects. Again, the children scored below the national 
comparative sample, with regard to Language and Cognitive Development, but equally on the 
Physical Health and Well Being dimension. The data was utilized to provide information similar 


to the above. projects. 


- Abbotsford in British Columbia has been involved in a project similar to those above and have 
completed two years. .Little information is yet available on their website. However, information 
provided indicates that the EDI is being utilized with regard to community mapping and provides 


information to help mobilize community resources similar to the previous projects. 


The Prince Edward Island study according to phone contact is the only study which has 
implemented the EDI province wide in all kindergarten classes. However, further information 
regarding this project was not received. According to a phone contact in New Brunswick, PEI 


has implemented a Kindergarten performance assessment in all kindergartens in the province as 
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a screening measure for teachers to make further referrals. Further specific information was 


unavailable at the time. 


In a communication from the research coordinator on April 15, 2003, regarding a project in New 
Brunswick, she indicated that they administered the EDI in 2000/2001 to 350 kindergarten 
children. She reported that they have just finished analyzing the data and are in the process of 
releasing this information to schools. They have now just finished collecting a second data point 
and once this is complete they will meet with interested parties in the communities to develop an 
action plan to support their young children’s learning. The ED] information along with the other 
research information they believe will provide baseline data to better focus programs and 

services to families. She indicated that there was some move in the province to attempt to use the | 
EDI data for assistance to teachers in instructional planning although this was not the case at 


present. 


The information acquired to date indicates that the data collected through the EDI in sites across 
the country has not been utilized to make educational programming decisions nor has it been 
used to assist teachers in adjusting instructional emphasis or approach in kindergarten or grade 
one classrooms. It would appear obvious that the combined data collected including the “Who 
am [I ?? and behavioral outcomes information on kindergarten children at these sites may indeed 
be valuable information which could be extremely helpful for educational programming for 
individual children demonstrating particular needs as identified by teachers and parents. Our 
current understanding is that this has not been the purpose of the previous studies. In contrast, 
Alberta Learning is interested in exploring not only how the EDI can focus on family and 
community resources, but also how it may be helpful for individual classroom educational 


programming. 
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Alberta Learning initiated a pilot project with 5 school jurisdictions in the fall of 1999. The 
purpose of this project was to 1) test the appropriateness of the Early Development Instrument as 
a province wide measure of Readiness to Learn, as in preparedness to learn; 2) to assess the 
applicability and usefulness of the information generated from the EDI to schools and 
jurisdictions for predicting cohort performance in early grades in school as well as for program 
planning; 3) to examine costs, inservice process and organization required to support use of EDI 


in schools. 


The study reported here addresses purposes one and two with respect to this pilot project. Three 


specific questions were addressed: 


Question 1: Was the support and organization provided during the pilot project adequate for _ 
use of the EDI in schools? 


Question 2: Was the data generated from the EDI and provided to schools useful in predicting 
cohort performance in the early grades in school as well as for teachers program 


planning? 
Question 3: Is the EDI appropriate as a province wide measure of readiness to learn?. 


The above questions were addressed through semi-structured interviews conducted by phone. 
Participants were teachers, co-coordinators and administrators who had been involved in the 
pilot study. Lists of names and school phone numbers of teachers and administrators who had 
been involved were forwarded to the research team by the project co-coordinator for each 
jurisdiction. Five school jurisdictions were involved: Calgary Public, Calgary Catholic, Rocky 
View, Aspen View, and Wetaskiwin. Stratified random sampling with replacement was used 
to select teacher and administrator participants. An attempt was made to obtain 25% of the 
original participants for phone interviews. All five project coordinators were interviewed. 


Some administrators were interviewed personally on site. Generally, it was difficult to arrange 
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phone interviews. Personal interviews with administrators were almost impossible to set up. 

It appeared that most principals were extremely busy. Repeated phone calls to administrators 
to set up convenient times to do a 20 minute interview were generally not returned. Of those 
few principals that we were able to be contacted by phone, five agreed to do a phone interview. 
In the end, one of those five principals called the day before the scheduled interview and said 
that she was not available, nor did she have any time the entire next week. In the end, two 
principals were personally interviewed on site, and two were interviewed on the phone. 


Teachers were easier to interview. In all, thirty-nine teachers were interviewed. 


All thirty-nine teachers had 4 or more years of teaching experience. Fifteen teachers had been 
involved for two years in the project. Twelve teachers reported special education training of at 
least 2 courses or more. Twenty-three teachers reported having ECS training. Eight teachers 
reported having taken no courses in the area of assessment while eighteen reported taking one 


course on assessment, and thirteen reported taking 2 or more courses in the area of assessment. 


A semi structured interview questionnaire was initially developed by the research team. This 
was then reviewed by a steering committee and revisions were then made. One was designed for 
teachers and one for administrator-coordinators. The phone interviews took from twenty to 
thirty minutes to complete. The data was then transferred into the computer and collated. A 
descriptive analysis was then conducted for the quantitative scales. The verbal answersto ~~ 
specific questions were recorded and transferred to computer. They were then collated and. 
themes were identified. The next section presents teachers’ results of the analysis, question by 


question. 
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Question 1: Is the support and organization provided during the pilot project adequate for 


use of the EDI in schools? 


In general, most of the teachers surveyed felt that the orientation sessions were ‘very useful’ (23 
out of 39). Teachers from all five jurisdictions commented on the thoroughness of the sessions, 
as well as the excellent support that they had received from their respective coordinators. 
Coordinators were available for consultation by phone or e-mail after the initial orientation 
session. In fact, teachers who had been involved for the second year suggested that future 
orientation sessions be streamlined to acknowledge the fact that they were familiar with the 
instrument. In contrast, new teachers could have a more in-depth orientation. A few teachers 
suggested they have the EDI at the beginning of kindergarten so that they could be “thinking 
about the questions throughout the year” before they actually did the assessments. 


When questioned about what they found most/least useful about the orientation sessions, 
teachers identified two main areas. First, teachers would have liked to have had the EDI 
instrument ahead of time so as to familiarize themselves as to its content. In this way, they could 
have brought questions to the orientation sessions. Second, teachers using the EDI for a second 
time would have appreciated an opportunity for some discussion or “troubleshooting”, such as 
second time to be tedious. Some teachers commented that future experiences using the EDI 
would probably be“‘less stressful” because they were now more familiar with the instrument. 
However, many teachers did not like the fact that they were not initially consulted about using 
the EDI. Many of them did not appreciate that they were “told that they had to do it, with little 


notice”. 


When asked about the adequacy of the support that they had received, most teachers (38 of the 
39) rated the support as ‘somewhat adequate to very adequate’. Although generally speaking the 
release time provided for teachers was reported to be ‘adequate’, about 25% of the teachers felt 


they needed more release time. Particularly problematic for many teachers was the completion 
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of the background information. Some teachers apparently went to “heroic” efforts to get at that 
information, as well as the signed permission forms. They called parents, sent out numerous 
reminder letters, and spoke to parents as they dropped their children off at school. Persistence 
appeared to be a key theme in this area. As a result, many of the teachers questioned the 
accuracy of the generated data. For example, many schools with a high ESL population got a 
very low rate of return of signed permission forms. In addition, many parents of children 
teachers believed were “‘at risk” academically did not sign permission forms. According to the 
teachers interviewed, parents cited such reasons as they did not want their children “labeled” and 
concerns about what was the purpose of the EDI, as well as how the information would be used 


in the future. 


In addition to the support and organization provided, the relative difficulty/ease of use of the EDI 
was an important ecnsideration in generating data. The teachers commented on how well the 
EDI was organized into the five domains; how easy it was to read; and how easy it was to enter 
_data. Most teachers (34 of 39) found the EDI ‘easy to very easy’ to use. Quite a few teachers 
commented on the fact that the information covered by the EDI was available in “bits and 
pieces” from their own reporting and classroom assessments; however, the EDI did a better job _ 
of depicting a “depth of coverage” on important domains of readiness to learn. In contrast, other 
teachers suggested that there was too much overlap in certain areas and that the EDI could be 
condensed. When questioned as to where the overlaps occurred, teachers could not remember 
because it had been a year ago since they had used the EDI. In addition, teachers’ comments 
again reflected the difficulty that they had experienced in filling out background information, 
such as if the child had attended a preschool. Many teachers recommended that if the EDI is 
implemented province wide, parents could fill this section out as part of the kindergarten 
registration process, as well as signing the permission forms. In general, teachers expressed the 
greatest difficulty with the time of year that they had to administer the EDI. For most teachers, 
spring (March, April) would have been a better time than May and June to assess the children. 
As a result, many of the comments focused on it being “too time consuming, given the time of 
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year”. 


Question 2: Is the data generated from the EDI and provided toschools useful for predicting 
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planning? : 


This question is best addressed in the context of the ‘larger picture’, namely what the teachers 
perceived to be the purpose and value of the EDI assessment. Responses ranged from “the 
government using the data to find out which children are ready for school, for statistical 
purposes, to someone at a university doing research”. Many teachers spoke of determining 
children’s “readiness skills” in various domains. What was less clear was teachers’ 
understanding that results are interpreted for groups of children, rather than individual children. 
In general, they did not see the implications in their classroom practice because they already 
“knew their students very well". At an individual classroom level, the EDI was not useful in 
providing new information to teachers as to their students’ readiness for grade one or further 
program planning. In other words, they may have missed the ‘larger picture’ that the EDI is 
trying to address, both at the macro and micro levels. Few teachers commented on the use of the 
data for comparing schools and communities, or program planning. This may have been an 
artificial artifact because many teachers did not receive the results of the report. When 
addressing the question as to whether this is the most efficient or best way of achieving this 
purpose, teachers’ opinions varied tremendously (38% of teachers indicated ‘yes’ to this 


question, 22% responded ‘no’ and 40% were ‘unsure’). 


The question regarding the utility of the EDI is somewhat difficult to answer accurately, because 


‘many of the teachers had not received the report generated from the EDI. In contrast, according - 


to the Year 1 Follow Up Survey Summary from Rocky View dated June 5, 2001, “100% of . 
teachers had heard the EDI results at the time of that survey”. The time frame from when the 
EDI was used and the follow- up report was the same as the current time frame. In the CBE 
Year 1 Follow Up Survey Summary, dated June 5, 2001, “84% of surveyed teachers had heard 
the results of the EDI pilot project for their district”. Of the small percentage (approximately 
5%) of currently surveyed teachers who had received the current report, most did not believe that 
the data was particularly helpful with regard to their classroom practice. In general, teachers felt 
that their own testing, report cards, and anecdotal accounts accurately depicted their students’ 


strengths and weaknesses. The EDI did not provide any useful or additional information for 
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them. A consistent theme centered on the fact that teachers knew their students very well, so that 
the results were no surprise. In other words, they could have predicted the results without having 
used the EDI. Several teachers stated that due to lack of parental permission, the cohort results 
were not at all representative of their classroom as many of the lower achieving students or 
English as a Second Language (ESL) students were not represented, and they in fact made up a 
large proportion of the classroom. Therefore, the EDI data was not value added for their 


classroom practice. 


There was, however, one exception. One jurisdiction where many of the teachers had either 
received the report or the information was shared with them by either the principal or coordinator 
had a slightly different perspective to offer. Some of the teaches had met with the grade one and 
grade two teachers (and the principal) in their school to discuss the needs in the school. They 
found the information useful in planning for their school’s needs. On the macro level, some 
teachers identified that the community-mapping portion may identify strong needs for 
programming and funding for schools. | 

It may be that without the benefit of having read the report many teachers did not really 
understand the purpose of the instrument and its implications for planning at the school, school 
board, community and provincial level. However, somewhat intuitively many of the teachers 


may have understood the purpose in their responses to question #3. 
Question 3: Is the EDI appropriate as a province wide measure of readiness to learn? 


Approximately 50 % of the surveyed teachers recommended that all schools and jurisdictions use 
the EDI.. However, there were some very explicit qualifications attached to its possible 

_ implementation province-wide. They only supported its use if it resulted in extra funding for 
schools, such as smaller class sizes, extra support such as aide time, speech/language services, 
and occupational therapy help. In addition, ESL support is not funded at the kindergarten level 
even though it was identified as sorely needed in some schools. Additional comments centered 
on inequities in various schools such as extra support being required in “needier areas”. A few 


teachers went so far as to suggest breakfast programs for hungry children coming to school. 
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In response to the question “Would you recommend that all schools and jurisdictions use the 
EDI?”, three of the five coordinators responded ‘no’. Some of the reasons to the question “why 
not?” were: “too labor intensive, incredibly expensive for release time, too time consuming, 
questions usefulness of community mapping and expense because of the large community 
growth each year; not aligned with curriculum, Ontario has Junior -Senior kindergarten or some 


demographics may not be applicable to the Alberta context”. 


Hypothetical Question 


The teachers who had not read the report were also asked if their views might change had they 
had a chance to read the report. Generally, the teachers still believed that there would have been 


little, if any, new information on their students. 


When asked the question “What particular purpose was the EDI data used for?”, one coordinator 
indicated that “it was not used for any particular purpose”; one commented on “its use for more 
instructional ideas for inservice and to have teachers think about the social/emotional aspects of 
children not just the academic but reflect on the whole child”; one reported that the school used it 
" to discuss the community mapping and its connection to SES; and the two remaining 


coordinators said that they were not using it directly to make any decisions. 
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ANALYSIS OF ADMINISTRATORS’ RESULTS 


Principals 


Two principals were interviewed personally on site. Three principals from two other © 
jurisdictions were interviewed on the phone. None of the 5 administrators (one principal had a 
husband who is also a principal and involved with the EDI) knew that the EDI had been 
administered in their school. They had vague recollections. It appears that AB Learning worked 
directly with the coordinators and it was up to the coordinators to communicate with principals. 
Three principals had not seen the report until March 17, 2003 at which time the coordinator 
shared the results (only 7 of the 17 principals invited actually attended). One of the principals 
has yet to receive a copy of the report. All five principals indicated that the communication 
process from AB Learning would have to be improved if the EDI was to go ahead. Four of the 
five rated the support they received during the project as ‘very inadequate’. The fifth principal 
was more generous and rated it as ‘somewhat adequate’, based more on the information that he 


had received after the 1999/2000 pilot project. 


One principal did not endorse its implementation, stating that “it was a waste of time and effort”. 
She stated that the results were not surprising. She would have predicted them based on her 
knowledge of the school population and community. The other principal said ‘yes’ with the 
qualification that “AB Learning take charge and identify the purpose and usefulness of the ED?’ 
before implementing it. Perhaps, it would be worthwhile if it is used for program planning, PUF 
grants, funding, Student Health Partnership for occupational therapy and speech/language 
services, etc. Another principal recommended that the EDI be implemented (again based on the 
1999/2000 results) if the EDI results “translated into dollars”. He commented on the fact that 
the EDI might provide some useful longitudinal data if the students were followed into grade 1 
and 2. The EDI might also be useful in changing teaching approaches if warranted. The results 
from the Social/Emotional domain might be particularly helpful in addressing certain teaching 
styles. In contrast, one of the other principals indicated that the information from this year’s 
report was not useful, yet the data from the previous one was. It appeared that the lack of 
timeliness in receiving the information from the current report was particularly problematic. 


Three of the principals rated the utility of the data provided as ‘not at all useful’ or ‘not very 
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above administrator data must be interpreted with extreme caution because of the few 


administrators actually interviewed. 


Generally speaking, the few principals that we were able to talk to when attempting to set up 
interviews stated that they had not had a chance to read the report. It appears that because the 
coordinators were in contact with the kindergarten teachers, the principals were perhaps not as 
involved as they could have been. It might be fair to suggest that the situation appears to echo 
sentiments expressed in the Follow Up Survey Summary from Rocky View dated June 5, 2001, 
namely, “administrators were in the dark about the whole thing” (p. lof 3). 


Coordinators 


All five co-coordinators were interviewed. The results of those interviews suggest that they did 
not feel that the EDI was the best or most efficient way of achieving the purpose as they saw it. 
Questions were voiced regarding the accuracy of the EDI, difficulty in acquiring background 
information and bias of the sample due to missing data from students not included, as well as the 
validity and reliability of teachers’ perceptions were voiced. They did feel that the support . 
provided was ‘somewhat adequate’. When asked if anything could have been done to better 
support them during the project, responses revolved around clearer communication from Alberta 
Learning with regard to receiving information and how it was to be used, including the follow-up 
process. These coordinators did rate the data as being “somewhat useful’; however, they . 
indicated that the receipt of this information was not timely, but too late. One indicated early 
fall, another one indicated that June would have been a better time for receipt of the data. With 
regards to how the EDI data was provided to teachers and schools, it was indicated that it was 


shared with principals and not sure from that point on. 


With regard to the questions, “How is the EDI data currently being used by teachers and 
schools” the majority of co-coordinators (4 of 5) reported not being sure as to how it was being 
used. Three of the five coordinators rated the information provided to them from the EDI as 
being ‘somewhat useful’, two rated it as ‘not at all useful’. They reported that the data was 
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provided to principals in the report that they had received from AB Learning. In some cases, it 
was shared with all principals in one jurisdiction and only some principals in other jurisdictions. 
In other words, principals were invited to attend a meeting, but many chose not to attend because 


of other commitments. 


Regarding the question, “How would you see the EDI data being used in the future if you had the 
opportunity?”, one coordinator indicated that it could be used as “hard evidence” to support 
needs in the community and schools; one suggested that it could be used “to plan programs, 


funding to access speech and O.T. services”; two replied, “it depends on the purpose”. 
Teachers’ Comments 


At the end of the sinvey: teachers were invited to add any comments that they felt were 
important to be shared. Concerns centered on issues of potential misuse. of confidential 
information, perceived “teacher resistance” to using the EDI, and interest in the community 
mapping portion by those few teachers who had had an opportunity to read the report. Some 
teachers had additional suggestions with regards to the background information. For example, a 
parent information package explaining how the results will be used might be helpful. Also, those 
communities with a high percentage of ESL families might benefit from having the information 


in their own language. 


Some teachers commented on the fact that the EDI made them more aware of children’s physical 
well-being in that they took this for granted and yet this was a domain of high need for some 
children. Other. teachers questioned whether we were expecting too much of kindergarten - 
children, that is, “expecting them to be so developed when they are not ready”. A similar 
sentiment was echoed by another teacher when she said, “that Kindergarten was becoming too 
academic with its emphasis on early literacy. It needed to continue to be play-centered”. There 
were quite a few teachers who questioned whether the EDI could be used to change the 
kindergarten entry age, moving it to perhaps having to be five years old by the.end.of December 
rather than the end of February. One teacher believed that she spoke for many of her colleagues 


when she emphatically stated that “kindergarten teachers are very frustrated, ...the significance 
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corrective learning”. Certainly, nationally, as well in the piloted sites in Alberta, older 
kindergarten children performed better than younger children on all five domains. Concerns 
were also voiced around the fact that kindergarten was not mandatory in all provinces, including 


Alberta. Therefore, how do we address the number of children who did not attend Kindergarten? 


Another problem area centered on the fact-that this survey was being conducted so long after the 
EDI implementation, a year in most cases. In future, any follow-up surveys should be conducted 
shortly after the implementation while the information is still fresh in teachers’ minds. For many 
teachers, the question as to whether all schools and jurisdictions should use the EDI was 
particularly problematic because they could not make a recommendation based on having 
received no results. One teacher wondered if the results could be shared with parents, such as at 


a Parent Council meeting. 
Administrators’ and Coordinators’ Comments 


Administrators and coordinators were also invited to share any additional comments: Most 
noteworthy was the information regarding the Program Enhancement Project (PEP) funding for 
schools in a low socio-economic area. At the suggestion of one of the interviewed principals, 
one of the PEP facilitators was interviewed on the phone. | 


The next section provides a summary of that interview. 


The Program Enhancement Project 


One of the facilitators agreed to a telephone interview as a result of her principal suggesting it in 
response to the question whether the EDI information was available in other ways. This was the 
first mention of this program, although the same question had been put to teachers, coordinators 
and other principals. The facilitator works under the guidance of the early education coordinator 
(interviewed in this report). The Brigance K screening form is used as the screener of all 


children entering Kindergarten in this school. Classroom observations also form part of the 
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screening process. The children are then classified into average, mild/moderately or severely 


delayed categories. 


The facilitator then organizes the children into somewhat homogeneous groups of 3-4. She 
worked with them 2-3 times per week on a pull-out basis for 45 minutes each time. During these 
times, she can also informally assess their gross and fine motor skills and as a result make 
appropriate referrals for occupational therapy assessments. The same is true of her observations 
of their language development. Appropriate referrals are also made to speech language 
therapists. The facilitator works on early literacy skills primarily using the “Animated Literacy” 
program, as well as working on early numeracy skills. Social skills are a focus as well. Because 
in her opinion, all children can benefit from early intervention, all children are included for 
shorter amounts of time and help. These more heterogeneous groupings encourage peers to help 
each other, in addition to improving social skills. According to the facilitator, these more 
heterogeneous groupings have been very helpful in “bootstrapping” less ready children’s 
understandings. The children are then assessed at the end of kindergarten using the Brigance K 
screener. This model may well be worth further investigation and development into the wider 


Alberta context. 
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Purpose of the EDI and the Rapid Change in Alberta Communities 


It is not the items that are of concern for assessing readiness but rather the purpose for which the 
instrument was developed and how it can be validly used. Given the rapid change in many communities 


in Alberta the mean scores used would be outdated almost as soon as the information was gathered. 


If this instrument was to be used for readiness there would need tobe extensive norming of the 
individual data to make it useful to the community. Also given the diversity of the communities and 
schools, the expectations of Grade One entry is important. In one northern school in Alberta it was 
stated that all children should know 22 of the 26 letters and at least 16 sounds to enter Grade One. Yet, 
in an inner city school in central Alberta, children are encouraged to learn the alphabet but this is re- 
taught in grade one and if not accomplished by January, intervention/remediation is undertaken. This 
type of diversity within Alberta makes norming a difficulty. If one’s intent was to have a standard 
curriculum with specific entry requirements and if the child cannot meet them then, remediation would 


occur when this is possible. 


Moreover, if one is to establish readiness at the community level, Love, Aber, & Brooks-Gunn (1994) 
suggest the following requirements: - 
1. assess all key dimensions relevant to the readiness goal 
focus on the collective status of entering Grade One 
rely primarily on existing instruments 
incorporate multiple modes of assessments 
incorporate multiple perspectives in the assessment 
be adaptable to local circumstances 


be appropriate for diverse cultural and racial/ethnic groups 


Gor a oO as ae ae 


balance positive and negative indicators of the readiness dimensions and be ready for 


implementation 
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Based on our findings, relying primarily on existing instruments is not recommended because these 


measures do not have adequate validity. é 


In the end, should children be ready for school or should schools be ready for children? Children come 
to us with various abilities, skills and experiences. It is up to the education system to find out where 
these children are and provide appropriate programming to assist them in being successful in the 
educational environment that we have created. This can be accomplished by assessing where the child 
is, What is needed, and then set out appropriate programming. The question is how to do this in the most 


effective and caring fashion possible. 


Limitations of our Study 


1. A limitation of this study was the limited number of principals who were interviewed. Of 
the ten to fifteen principals whom we attempted to contact for interviews, only 5 
interviews were conducted. Repeated calls did not improve the number of principals 
contacted. If we had more time, more of the principals, and possibly teachers would by 
now have had a chance to read the report. In all cases, it was not until we contacted the 
coordinators that the information was disseminated. In general, the data presented to the 
teachers was not useful for instructional purposes. However, caution in interpreting the 
‘usefulness of the data’ must be exercised because many of the teachers and principals 
did not get a chance to read the report. Questions regarding ‘the ease of use and. 

- adequacy of support’ are interpretable and may be considered representative of teachers’ 
opinions. 

2. Another limitation voiced by many of the teachers and coordinators was the fact that the 
survey was such a long time after they had used the EDI. A few of the teachers even 
reviewed the EDI before they would agree to an interview. Some of the coordinators 
asked for more time to review the report before giving the interview. This speaks to the 


professionalism and conscientiousness of both the teachers and coordinators. 


3. With regards to the actual number of participants in the 2001-2002 EDI project, there is a 


large discrepancy in most jurisdictions of actual participants when compared to expected 
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in fact 2,770 valid returns were ice “According to the EDI developers, the aetial: _ 
number of students is based on the number of valid EDI instruments returned to 
McMaster University. For all five participating jurisdictions, the number of EDI forms 
returned was substantially higher than the number of valid returns. These numbers are 
based on signed parental permission forms. As indicated in this report, some teachers 
reported low rates of return, as low as 50% of students in their class. Many teachers 
commented on the less than ideal rates of returns. As a result, the data may not be as 


representative of the population. 


. The determination of the EDI cut-offs for the national cohort may be problematic. It is 
our understanding that the students were not randomly selected demographically. Asa 
result, the national cohort may not be represéntative of Canadian students. In addition, 
potentially somewhat problematic is establishing the “low score” threshold at “below the 
10" percentile”. As described on the EDI, the “below the 10" percentile” is the cut-off 
for vulnerability status for school readiness (Hertzmann, Kohen, McLean, Evans, and 
Dunn, 2001). Although not completely arbitrary, this cut-off may be too low or too high 
depending on the population assessed. A further refinement or consideration of the use 
of the EDI would be to address the cut-off. As it stands, the current results comparing the 
Alberta to national cohorts as well as using the set 10% cut-off may not be adequately 


representative of the assessed Alberta students. 


The following sectron presents the results of the implementation of the EDI in 2001-2002 in five 
Alberta jurisdictions. 
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ALBERTA EDI REVIEW 


The EDI was completed by teachers for all kindergarten children whose parents had signed 
permission forms in selected Alberta communities. It is unclear from the information provided 
whether the schools in these communities volunteered or were selected. In the 1999/2000 school 
year it was completed for 2800 students in Alberta, and in the 2001/02 it was completed for 
2,770 students of which 2,340 Alberta students were actually used. First, the results from the 
Spring 2002 reports are presented by school district. For all five participating jurisdictions, the 
number of EDI forms returned to McMaster was substantially higher than the number of valid 
returns. Next, the Spring 2001 report based on the results from the 1999/2000 school year is 


briefly summarized. 


Calgary Board of Education 


The EDI was completed for 1173 Kindergarten students in the Calgary Board of Education 
’ (CBE) in the 2001/2002 year. In all sites across Canada in 2000/2001, the EDI was completed . 
for over 25,000 Senior Kindergarten level students (national cohort). 


On average, the group of kindergarten children in the CBE is doing just as well as children in last 
year’s national cohort (Spring 2002 report). The CBE’s means are slightly lower than the last 
year’s cohort in Physical Health and Well-Being and Communication Skills and General 
Knowledge domains, but slightly higher on the Social Competence, Emotional Maturity and 
Language and Cognitive Development. During the 2001/2002 school year, 400 out of 1173 
(34%) had problems in at least one readiness to learn domain, and 179/1173 (15.2%) had | 
problems in at least two domains. The comparable percentages for the entire last year’s cohort 


are 26.3% and 13.1% (with ranges 20.2% to 29.2%, and 11.2% to 15.2%, respectively. 


Of the 1412 kindergarten children assessed, 195 children with special needs were excluded, and 
10 were missing, resulting in a total of 1173 students. This number did not account for the 
number of children whose parents did not sign the release forms for their children to be assessed. 
According to teachers’ comments during the survey, the number of returned and signed release 


forms, ranged from as low as 50% of children to as high as 90% of the forms. 
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As described on the EDI, the “below the 10" percentile” is the cut-off for vulnerability status for 
school readiness (Hertzmann, Kohen, McLean, Evans, and Dunn, 2001). Thus, those 
kindergarten students’ scores that fell among the lowest (poorest) ‘0% of scores encountered in 
the CBE were said to be “at risk” in terms of school readiness, with respect to the particular 
domain on the EDI. Therefore, theoretically, if children’s needs were evenly spread across the 
district, each school should have 10% of its students fall into this category. For example, in a 
school with 100 kindergarten students, it is to be expected that 10 students will score very low on 
one or more of the domains. An examination of the school reports showed, however, that this 
was not the case. The differences between schools in the CBE varied tremendously. The 
percentages of children whose scores were in the “below the 10" percentile” category were 
demonstrated by the following ranges. For the Physical Health.and Well-Being scale, schools in 
the CBE varied from 0-50% in how many children scored “below the 10" percentile”. For the 
Social Competence scale, this range was 0-46%, for Emotional Maturity 0-53.8%, Language and 
Cognitive Development 0-46%, and Communication Skills and General Knowledge 0-57.1%. 


Furthermore, students in some schools may score in the “below the 10° percentile” on more than 
one scale. The percentage of such students per school varied in the CBE from 5.9% to 75.0%. 
Theoretically, it is fair to expect that there will be students in such a category. However, if it ~ 
happens for more than 20-30% of the population, it is a good indicator that there are children 
with increased needs in the particular school and the particular community. According to the 
Spring 2002 report, 28/45 (62.2%) schools in the CBE had more than 30% of children with 
increased needs on more than one domain. According to the EDI developers, the scores on the 
readiness to learn domains can be used to identify these increased needs more closely for the 
particular school. Some of the associating factors can be examined by exploring the 
characteristics and resources of neighborhoods where children attending this school live. This 
kind of information should be accessible locally. For example, in the CBE, there are 28/45 
(62.2%) schools in which there were three or more children (30% or more) who scored in the 
lowest 10% on the Emotional Maturity scale. The “School Readiness to Learn Profiles” 
described these children as “almost always distractible and inattentive and acting impulsively; 


40 


they have regular problems managing aggressive behavior and temper tantrums, are occasionally 


worried or anxious, and are unable to show helping behavior towards other children”. 


Several comparisons were carried out (McMaster University) on the CBE data. The following 


are the results from the 2001/2002 report: 


Comparison 1: For all five domains of the readiness to learn at school, girls consistently scored 
better than boys. This is a consistent developmental characteristic across all sites where the EDI 
was implemented. 

Comparison 2: Children born earlier in the year scored better than children born later in the 
year. This is also a consistent developmental phenomenon: older children are more ready to 
learn at school than younger children. 

Comparison 3/4: The proportion of children with ESL status was about 36%. These children 
scored significantly higher than those without the ESL status in the Physical Health and Well- 
Being domain, and significantly lower in the Language and Cognitive Development and 
Communication Skills and General Knowledge domains. Similarly, children whose first 
language was English scored significantly higher than those whose first language was not 
English in the Language and Cognitive Development and Communication Skills and General 
Knowledge domains. | 

Comparison 5: Children who attended French Immersion comprised less than 3% of the 
population and thus the results here should be interpreted with extreme caution. ‘These-‘children 
scored significantly lower than children who attended regular English classes in the domain of ~ 
Emotional Maturity, and significantly higher on Language and Cognitive Development and 
Communication and General Knowledge. 

Comparison 6: Children with Aboriginal status represented only about 1% of the sample, and 
thus no statistical analyses were carried out. 

Comparison 7: Children with identified problems who attended early intervention scored 
significantly worse than the other children in the Physical Health and Well-Being domain. 
Comparison 8: Children who attended language or religion classes scored significantly higher 
than those who did not-in the domain of Language and Cognitive Development but lower in the 


Communication and General Knowledge. 
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higher on all dgmaains ocuupaed to those children who did not attend. 

Comparison 10: Children who attended child-care arrangements part-time scored significantly 
higher than the children who attended full-time in the domains of Physical Health and Well- 
Being and Language and Cognitive Development in this population of children. 


Wetaskiwin 


The EDI was completed for 133 Kindergarten students in the Wetaskiwin 
Regional Division No. 11 in the 2001/2002 year. In all sites across Canada in 2000/2001, the 
EDI was completed for over 25,000 Senior Kindergarten level students. 


On average, the group of kindergarten children in Wetaskiwin schools is doing just as well as 
children in last year’s national cohort. However, in the Language and Communication domain, . 
children in Wetaskiwin are doing considerably better. During the 2001/2002 school year, 37out 
of 133 (27.8%) had problems in at least one readiness to learn domain, and 16/133 (12.0%) had 
problems in at least two domains. The comparable percentages for-the entire last year’s cohort 


are 26.3% and 13.1% (with ranges 20.2% to 29.2%, and 11.2% to 15.2%, respectively). 


Of the 159 kindergarten children assessed, 25 children with special needs were excluded, and 1 © 
was missing, resulting in a total.of 133 students. This number does not account for the number 
of children whose parents did not sign the release forms for their children to be assessed. It is 
important to note that the total number of children from the Wetaskiwin Regional Division No. 
11 who received the EDI may not be representative of the total population of kindergarten - 
children in the Wetaskiwin school district. There were 4 schools with no questionnaires returned 


to McMaster University. 


As described on the EDI, the “below the 10" percentile” is the cut-off for vulnerability status for 
school readiness. Thus, those kindergarten students’ scores that fell among the lowest (poorest) 
10% of scores encountered in the Wetaskiwn were said to be “at risk” in terms of school 


readiness, with respect to the particular domain on the EDI. Therefore, theoretically, if 
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children’s needs were evenly spread across the district, each school should have 10% of its 
students fall into this category. For example, in a school with 100 kindergarten students, it is to 
be expected that 10 students will score very low on one or more of the domains. An examination 
of the school reports showed, however, that this was not the case. For the Physical Health and - 
Well-Being scale, schools in Wetaskiwin varied from 0-53.8% in how many children scored 
“below the 10" percentile”. For the Social Competence scale, this range was 0-34.6%, for 
Emotional Maturity 0-33.3%, Language and Cognitive Development 0-38.5%, and 
Communication Skills and General Knowledge 0-25.01%. 


Furthermore, students in some schools may score in the “below the 10" percentile” on more than 
one scale. The percentage of such students per school varied in Wetaskiwin from:18.2% to 
66.7%. Theoretically, it is fair to expect that there will be students in such a category. However, 
if it happens for more than 20-30% of the population, it is a good indicator that there are children 
with increased needs in the particular school. According to the Spring 2002 report, 5/8 (62.5%) 
schools in Wetaskiwin had more than 30% of children with increased needs on more than one 
domain. According to the EDI developers, the scores on the readiness to learn domains can be 
used to identify these increased needs more closely for the particular school. Some of the 
associating factors can be examined by exploring the characteristics and resources of 
neighborhoods where children attending this school live. This kind of information should be 
accessible locally. For example, in Wetaskiwn, there were 4/8 (50.0%) schools in which there 
were three or more children (30% or more) who scored in the lowest 10% on the Emotional 


Maturity scale. . 


Several comparisons were carried out (McMaster University) on the Wetaskiwin data. The 


following are the results from the 2001/2002 report: 


Comparison 1: For all five domains, girls consistently scored better than boys. 
Comparison 2: Children born earlier in the year scored better than children born later in the 


year. 
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on the Communication Skills and Geneial Knowledge domain than those who did not attend: - 
French Immersion. 

Comparison 4: Students with Aboriginal background scored significantly lower in the 
Communications Skills and General Knowledge domain than children with other backgrounds. 
Comparison 5: There was no statistical significance in the scores on any of the domains of 
children with identified problems who attended early intervention compared to children who did 
not attend an early intervention program. 

Comparison 6: There was no statistical significance between scores of children who attended 
language or religion classes and those who did not. 

Comparison 7: Children who did not attend preschool scored si gnificantly better in the 


Communication Skills and General Knowledge domain than children who attended preschool. 


Rocky. View 


The EDI was completed for 585 Kindergarten students in Rocky View School Division No. 41 in 
the 2001/2002 year. In all sites across Canada in 2000/2001, the EDI was completed for over 
25,000 Senior Kindergarten level students. 


On average, the group of kindergarten children in Rocky View schools is doing better in all . 
domains compared to last. year’s national cohort.(Spring 2002 report). During the 2001/2002 
school year, 130 out of 585 (22.2%) had problems in at least one-readiness to learn domain, and 
59/585. (10.1%) had problems in at least two domains. The comparable percentages for the entire 
last year’s cohort are 26.3% and 13.1% (with ranges 20.2% to 29.2%, and 11.2% to 15.2%, 


respectively. 


Of the 624 kindergarten children assessed, 30 children with special needs were excluded, and 9 
were missing, resulting in a total of 585 students. This number does not account for the number 


of children whose parents did not sign the release forms for their children to be assessed. 
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As described on the EDI, the “below the 10™ percentile” is the cut-off for vulnerability status for 
school readiness. Thus, those kindergarten students’ scores that fell among the lowest (poorest) 
10% of scores encountered in Rocky View were said to be “at risk” in terms of school readiness, 
with respect to the particular domain on the EDI. Therefore, theoretically, if children’s needs 
were evenly spread across the district, each school should have 10% of its students fall into this 
category. For example, in a school with 100 kindergarten students, it is to be expected that 10 
students will score very low on one or more of the domains. An examination of the school 
reports showed, however, that this was not the case. For the Physical Health and Well-Being 
scale, schools in Rocky View varied from 0-24.3 in how many children score very low. Forthe _ 
Social Competence scale, this range was 0-33.3%, for Emotional Maturity 3.8-33.3%, Language 
and Cognitive Development 2.5-33.3%, and Communication Skills and General Knowledge 0- 


16.0%. In addition, students in some schools may score “very low” on more than one scale: © 


Furthermore, students in some schools may score in the “below the 10" percentile” on more than 
_one scale. The percentage of such students per school varied in Rocky View from 10.5% to 
38.6%. Theoretically, it is fair to expect that there will be students in such a category. However, 
if it happens for more than 20-30% of the population, it is a good indicator that there are children. 
with increased needs in the particular school. According to the Spring 2002 report, 4/12 (33.3%) 
schools in Rocky View had more than 30% of children with increased needs on more than one 
domain. According to the EDI developers, the scores on the readiness to learn domains can be 
used to identify these increased needs more closely for the particular school. Some of the 
associating factors can be examined by exploring the characteristics and resources of 
neighborhoods where children attending this school live. This kind of information should be 
accessible locally. For example, in Rocky View, there were 11/12 (91.7%) schools in which 
there were three or more children (30% or more) who scored in the lowest 10% on the Emotional 


Maturity scale. 


Several comparisons were carried out (McMaster University) on the Rocky View data. The 


following are the results from the 2001/2002 report: 


Comparison 1: For all five domains, girls consistently scored better than boys. 
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year. 

Comparison 3: The proportion of children with ESL status was about 3%. There was no 
Statistical significance in any of the domains between the scores of thes children and the scores 
of those without the ESL status. 

Comparison 4: Children who attended French Immersion scored significantly lower than 
children who attended regular English classes in the domain of Emotional Maturity, and 
significantly higher on Communication Skills and General Knowledge. 

Comparison 5: Children with identified problems who attended early intervention scored 
significantly worse than the other children in all of the domains with the exception of Social 
Competence. It has been found that these children usually have significant deficits to make up 
and rarely do so by the end of the kindergarten year. 

Comparison 6: There was no statistical significance in the scores between children who 
attended language or religion classes and those who did not attend such classes. 

Comparison 7: Children who attended an organized part-time preschool scored significantly 
higher on the Physical Health and Well-Being, Language and Cognitive Development and 
Communication and General Knowledge domains. 

Comparison 8: Children who attended Junior Kindergarten scored significantly higher than — 
those who did not in Physical Health and Well-Being, and Communication and General 
Knowledge domains. 


Calgary Roman Catholic Separate School Division 


The EDI was completed for 449 Kindergarten students in the Calgary Roman Catholic Separate 
School Division in the 2001/2002 year. In all sites across Canada in 2000/2001, the EDI was 
completed for over 25,000 Senior Kindergarten level students. 


On average, the group of kindergarten children in the Calgary Roman Catholic Separate School 
Division is doing better in all domains compared to last year’s national cohort (Spring 2002 


report). During the 2001/2002 school year, 127 out of 449 (28.3%) had problems in at least one 


46 


readiness to learn domain, and 59/449 (13.1%) had problems in at least two domains. The 
comparable percentages for the entire last year’s cohort are 26.3% and 13.1% (with ranges 


20.2% to 29.2%, and 11.2% to 15.2%), respectively. 


It is important to note that the total number of children from Calgary Catholic who received the 
EDI may not be representative of the total kindergarten population in this division. There were 5 
schools who did not return questionnaires. In all of the 20 schools included in the analysis there 


were missing EDI questionnaires. 


The number of children with special needs that were excluded as well as the number of missing 
children’s surveys was not included in the report. This number does not account for the number 


of children whose parents did not sign the release forms for their children to be assessed. 


As described on the EDI, the “below the 10" percentile” is the cut-off for vulnerability status for 
school readiness. Thus, those kindergarten students’ scores that fell among the lowest (poorest) 
10% of scores encountered in Rocky View were said to be “at risk” in terms of school readiness, 
with respect to the particular domain on the EDI. Therefore, theoretically, if children’s needs 
were evenly spread across the district, each school should have 10% of its students fall into this 
category. For example, in a school with 100 kindergarten students, it is to be expected that 10 
students will score very low on one or more of the domains. An examination of the school - 
reports showed, however, that this was not the case. For the Physical Health and Well-Being 
scale, schools in Calgary Catholic vary from 0-4 1.7% in how many children score very low.: For 
the Social Competence scale, this range was 0-32.1%, for Emotional Maturity 0-41.7%, 
Language and Cognitive Development 0-43.8%, and Communication Skills and General 
Knowledge 0-40.0%. 


Furthermore, students in some schools may score in the “below the 10 percentile” on more than . 
one scale. The percentage of such students per school varied in Calgary Catholic from 7.7% to 
58.3%. Theoretically, it is fair to expect that there will be students in such a category. However, 
if it happens for more than 20-30% of the population, it is a good indicator that there are children 
with increased needs in the particular school. According to the Spring 2002 report, 10/20 


ve 
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more than one domain. According to the EDI developers! the scanty on the seadiness to learn 
domains can be used to identify these increased needs more closely for the particular school. 
Some of the associating factors can be examined by exploring the characteristics and resources 
of neighborhoods where children attending this school live. This kind of information should be 
accessible locally. For example, in Calgary Catholic , there were 7/20 (35%) schools in which 
there were three or more children (30% or more) who scored in the lowest 10% on the Emotional 


Maturity scale. 


Several comparisons were carried out (McMaster University) on the Calgary Catholic data. The 
following are the results from the 2001/2002 report: 


Comparison 1: For all five domains, girls consistently scored better than boys. 

Comparison 2: Children born earlier in the year scored better than children born later in the 
year. 

Comparison 3 & 4: The proportion of children with ESL status was about 14%. These children 
scored significantly lower than those without the ESL status in the Communications skills and 
General Knowledge domain. 

Comparison 5: Children who attended French Immersion comprised about 3% of the 
population. There was no statistical significance between the scores of children who attended 
French Immersion and those who did not in any of the domains. 

Comparison 6: Children with identified problems who attended early intervention scored 
significantly worse than the other children in the Communication Skills and General Knowledge 
domain. 

Comparison 7: There was no statistical significance between scores of children who attended 
language or religion classes and those who did not. 

Comparison 8: Children who attended preschool scored significantly higher in the Language 
and Cognitive Development domain compared to those children who did not attend. 
Comparison 9: Children. who attended Junior Kindergarten scored significantly lower in the 
Physical Health and Well-Being domain than those who did not. 
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Comparison 10: Children who attended child-care arrangements part-time scored significantly 
higher than the children who attended full-time in the domains of Physical Health and Well- 
Being and Language and Cognitive Development in this population of children. 


Aspen View 


The EDI was completed for 99 Kindergarten students in the Aspen View Regional Division No. 
19 in the 2001/2002 year. In all sites across Canada in 2000/2001, the EDI was completed for 
over 25,000 Senior Kindergarten level students. | 


On average, the group of kindergarten children in the Aspen View Regional Division No. 19, 
compared to children in last year’s cohort, is doing better in the Emotional Maturity domain, 
worse in Physical Health and Well-Being, and Communication and General Knowledge 
domains. They are doing the same in the Social Competence and Language and Cognitive - 
Development domains. During the 2001/2002 school year, 29 out of 89 (32.6%) had problems in 
at least one readiness to learn domain, and 17/89 (19.1%) had problems in at least two domains. 
The comparable percentages for the entire last year’s cohort are 26.3% and 13.1% (with ranges 


20.2% to 29.2%, and 11.2% to 15.2%, respectively. 


Of the 99 kindergarten children assessed, 8 children with special needs were excluded, and 1 was 
missing, resulting in a total of 99 students. This number did not account for the number of 
children whose parents did not sign the release forms for their children to be assessed. School- 
level results cannot be examined against the district means because there was only one school 


involved. 


The next section reviews.the Year 1 follow-up report in Calgary and Rocky View. 


Report- Year | Follow-Up 


In the spring of 2000, approximately 2800 children (aged 4.5-6 years) from 64 kindergartens in 
the Calgary Board of Education and Rocky View were assessed using the EDI. The Calgary and 
Rocky View Readiness to Learn Community Mapping Project was comprised of two 


components: the EDI and the Community Resource Inventory (CRD. 


Outcomes on the EDI varied by community and by domain, but overall students did 
comparatively well on all dimensions of readiness to learn. Overall, communities performed 
best on the Language and Cognitive Development domain, and not as well on the Physical 
Health and Well-Being domain. 


The CRI for Calgary was conducted in the spring of 2001. The Rocky View information was not 
included at the time of the November 2001 report. The CRI examined the study area — 
communities with regards to their physical and socio-economic characteristics as well as 
available community resources. In comparing the characteristics of 68 Calgary communities to 
the EDI results, it was reported that socio-economic characteristics were more strongly 
correlated with readiness to learn than physical characteristics or available resources. In 
addition, some community characteristics were more strongly correlated with one or two 


domains, rather than with overall readiness to learn. 


The report focused primarily on the correlations between EDI scores and community 
characteristics in those communities that were found to be most vulnerable. A gradient of “at 
risk” status was developed to investigate differences between communities. With the exception 
of one Calgary community (37% in the 10™ percentile), the 16 Calgary communities highlighted 
as “vulnerable” had more than 40% of assessed students falling in the 10™ percentile on at least 
one domain. The CRI was then used to analyze Cluster A communities where more than 20% of 
the EDI results fell in the 10 percentile and as a result their students were most vulnerable with 
regards to readiness to learn as measured by the EDI. Neighborhood characteristics, particularly 


socio-economic characteristics such as neighborhood affluence, have a critical impact on 
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preschooler’s readiness to learn (Kohen, Hertzman and Brooks-Gunn, 1998). Neighborhood 
affluence is assessed in terms of economic well-being, social well-being, as well as physical 
well-being and safety. In this report, socio-economic characteristics were more strongly 
correlated with readiness to learn than physical characteristics or available resources. Poverty 


and unemployment were used as the two main types of economic well-being indicators. 


EDI Linkages 


According to the 2001 report, “a comparison of the above ‘economic well-being’ indicators and 
the results of the five EDI dimensions confirms the expected pattern” (p.19). Overall, Cluster A 
communities in Calgary tended to have the highest percentage of low income families, the 
highest percentage of children in families receiving income supplements and the communities 
with the highest unemployment rate. In addition, education levels of families are considered a 
critical contribution to the socio-economic environment of communities. Namely, adults in 
communities with higher levels of education tend to be more likely to be employed, are less 
likely to live in poverty, and more likely to serve as role models. The education of parents has 
been shown to be related :to the development of children (Connor, 2001). A well recognized 
gradient is the Socioeconomic Gradient. The pattern wherein risk increases in a stepwise fashion 
as one descends the socioeconomic ladder is known as a gradient. For example, a . 
socioeconomic gradient that compares children’s literacy levels:‘with parents’ level of education 
demonstrates how critical parents’ level of education is to the child’s development. However, 
overall according to the authors of the 2001 report, the linkages between community resources 


(CRI) and readiness to learn status were not as informative as originally anticipated. 


A surprising result of the CRI was the fact that ESL and immigrant status did not appear to be 
directly correlated with readiness to learn domains, including Language and Cognitive 

‘Development and Communication Skills and General Knowledge. The authors of this report 
would like to suggest that this population might have been underrepresented as seems to be 
indicated by anecdotal reports from many of the current surveyed teachers and most 


coordinators. Namely, many parents of ESL children did not sign the required permission forms 
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the November, 2001 report. 
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CONCLUSIONS 
SURVEY RESULTS AND ALBERTA REPORTS 


If as stated, teachers, coordinators and principals could have predicted the results without using 
the EDI, it begs the question of the usefulness of the EDI. At the heart of the concern is whether 
the purpose of the EDI instrument as stated by the originators “to provide a developmentally 
based (as opposed to curriculum-based) data base, applicable in every province, and interpretable 
at the group level for populations of children” (Janus. & Offord, 2000, p. 73) is AB Learning’s 
purpose. Is it to identify problem areas for groups of children with all interventions such as 
literacy, speech/language and occupational therapy, to name but a few, having a central universal 
component as opposed to identifying individual children “at-risk” who are then targeted for 
specific intervention programs? For example, the Program Enhancement Project appears to 
target schools in low socioeconomic areas and provide early kindergarten intervention using 
facilitators working with small groups of identified students at-risk for school difficulties. It 
appeared from the limited conversation with one of these facilitators that more than the 10™ 


percentile cut-off threshold used by the EDI would apply in that particular school. 


Furthermore, 50% of teachers and coordinators who supported the implementation province- 
wide qualified their endorsement. They would only support it if the EDI resulted in additional 
funding for supports and program planning. Namely, the data would be used to support students’ 


learning. 


With regards to community mapping, interesting jurisdictional differences arose. For example, 
areas such as Calgary and Rocky View that are experiencing unprecedented growth and rates of 
change wondered how useful the information would be within a year’s time. Certainly, the 
difficulty of interviewing teachers and principals was noteworthy in these areas. Both areas had 
experienced large turnovers in staffing. In contrast, smaller more rural communities felt that the 
community mapping would not provide any new information. Although these comments are 
based on a very small sample, they are worth noting. Wetaskiwin and Aspen View were already 
involved in inter-agency meetings. The coordinators regularly attended these meetings sharing 


information and resources. They appeared to work within a very different model from larger 
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community mapping portion aiailable already. Community mapping is a labor-intensive process 
(costly?). Agencies such as public health, daycares and preschools, census Canada, community 
recreation etc. already collect information on children and their families. For example, if we 
already know that low socioeconomic status is a predictor of children being less ready for school, 


then is there an effective way to access this information? » 


With regards to the actual use of the EDI instrument, the gathering of signed permission slips, as 
well as the entering of background information was particularly problematic. Several teachers 
and coordinators suggested that if AB Learning implemented the EDI province-wide, then 
parents would be expected to fill these out as part of the ECS registration process. Concerns 
were also raised regarding ESL parents. Were there parental letters in various languages 
available from Ontario where the EDI had been extensively piloted? The letter would explain 
the purpose of the instrument as well as to how it might be used in Alberta. Confidentiality was 


a critical element in this process. 


Although some of the teachers saw some merit in the EDI there were administrative and 
communication difficulties that must be cleared before recommending the implementation across 
Alberta. Teachers generally felt that they did not understand the rationale behind the use of the 
EDI. In particular, they would have appreciated some follow up information in a timelier 
manner. As it stands today, many teachers surveyed have not received the results. Coordinators ° 
were waiting for information from Alberta Learning as to how and when to distribute the results. 
It was sometimes*not until we contacted the coordinators asking them for names and phone - 


numbers that the information began to be disseminated. 


Of significant importance is the problem of lack of validity data, particularly concurrent and 
predictive validity, with respect to the EDI as well as the somewhat negative response regarding 
participants’ views with regard to province wide implementation. It is not recommended that the 
EDI be implemented province wide at this time. It is recommended that until further evaluation 


of its psychometric properties, as well as methods of implementation is undertaken, this should 
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not be attempted. The following section presents the results of the implementation of the EDI in 
2001-2002 in five jurisdictions in Alberta. 
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The concept of “readiness” is clearly a controversial one, as is its mneasuremient: The most 
important consideration would appear to be “to what purpose” is this concept being used and 
how to best assess it. The issue of “group” consideration versus “individual” is also one which 
needs to be addressed in the use of any particular measure. The EDI was developed as a group 
measure and is to be interpreted in this light. The reliability and validity of this instrument even 
as a group measure to our knowledge has not been clearly established which raises questions 
about its appropriateness as a measure of “at risk”. The lack of appropriately validated cut off 
scores or levels is also a concern with regard to comparative analysis of different groups. The 
comparative group (national group) which has been used lacks representativeness of the 
population and yet has been used to make these comparative analyses. The issues of reliability 
-and validity particularly concurrent and predictive validity is even more of a concern when this 
instrument is used to make individual and group classroom programming decisions. This 

_ instrument could be further refined (e.g. add in some auditory analysis skills, and other 
metacognitive skill assessment) and then conduct further studies to more clearly establish its 


concurrent and predictive validity, as well as its ecological validity with regard to classroom use. 


The results of the interview survey indicated that with respect to question #1 which addressed the 
adequacy of support and organization provided, the participants viewed this as “very adequate’. 
This included the orientation sessions which were viewed very positively as well. The release 
time provided to conduct the assessments was viewed by participants as extremely helpful. The 
only adjustment was to have the instrument ahead of orientation session to familiarize 
themselves and more discussion of case studies for those teachers involved in year 2, having 
been also involved in year 1. The vast majority of participants found the EDI ‘easy to use’; 


however, they would prefer to do it in March/April of the year. 


Question #2 required participants to respond regarding their perceptions of the ‘usefulness of the 
data generated’ from the EDI with respect to classroom programming, working with individuals 
and predicting cohort performance. The results of the semi-structured interviews suggest that the 
vast majority of participants did not find the data useful to them with regard to their particular 
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classrooms. With regard to cohort performance, somewhat more positive views were evident 


particularly with regard to school to school or system to: system comparisons. 


The data is clear in indicating that administrators appeared often to be unaware of the use of the 
instrument in their schools or of the data provided. It may have been more helpful to have 
school administration involved in all aspects, including the orientation sessions for teachers. 


This did not appear to have happened. 


Question #3 required participants to indicate whether they felt the Early Development Instrument 
should be implemented province wide. The results of the interviews indicated that about half the 
teachers indicated yes, but that conditions were attached to this. The majority of 
coordinators/administrators indicated this would not be appropriate. The few who indicated a 

‘ tentative yes also attached some conditions. Responses of “it depends on the purpose” were also 


voiced. 


A significant number of difficulties were encountered during the interview process. The most 
significant was that a large number of participants had not had the opportunity to review the data 
as they had not received it, even though jurisdictions received it some time earlier. This made it 
extremely difficult to obtain valid evaluations particularly with regard to the utility of the data 
provided to schools. The apparent lack of real involvement by school administration made it 
almost impossible to obtain a valid picture of their perceptions regarding the use of the EDI 
information in their schools. The results therefore must be interpreted with caution including 
EDI results distributed to schools, as a large amount of missing data makes classroom, school or 


jurisdiction comparisons impossible. 
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1. The EDI lacks sound psychometric properties to warrant using it to make predictions about 
groups of children’s readiness for school and more specifically with regard to individual or 


classroom use. 


According to one of the developers of the EDI, Dr. Janus, (personal communication) there are no 
published studies with regards to its reliability and validity. There is some evidence from the 
research done on the EDI that it has reasonable inter-rater reliability. However, concurrent and 
predictive validity has not been established. As a result, further evaluations would need to be 


conducted to improve its psychometric properties. 


2. Even if the EDI’s psychometric properties are addressed, the results of the EDI surveys 
suggest that the teachers would have been able to make the same predictions that the EDI made. 
Teachers generally felt that the EDI provided no new information on their students. However, 
some teachers did acknowledge that had they had the report to read there might have been some 
new information regarding the community. Coordinators voiced similar views. Coordinators 
along with the few administrators interviewed, expressed being tentative as well in - 


recommending its implementation province-wide. 


3. Of paramount importance is the question of whether the purpose of the EDI as a group . 
measure meshes with Alberta Learning’s goals in how they might use it. 

The EDI is a group measure and was not designed to identify individual students at-risk for 
school difficulties. In addition, a key goal of the EDI is the Community Resource Inventory 
which can be used to mobilize existing or new community resources. 

For example, it is a well-established fact that low socioeconomic status is an important predictor 
of children’s school readiness. As a result, we believe in the idea of a readiness assessment to 

_ identify groups of children “at risk” for school difficulties. Perhaps, further refinement of the 
EDI might be warranted. 
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4. The further refinement of the EDI could incorporate some auditory analysis skills assessment 
and other metacognitive skills assessment such as cognitive monitoring and behavioral 
monitoring, etc. Further emphasis on the development of the social, emotional and behavioral 


dimensions we see as being important as well. 


Research has clearly indicated that auditory analysis skills and the use of metacognitive 
strategies have been shown to be good predictors of reading success. An alternative suggestion 
is to develop a new instrument using the five domains but doing a factor analysis to eliminate 
possible overlaps among domains, and thereby shortening the number of items to be 
administered. We like the fact that the EDI has incorporated measures to assess children’s 
physical and emotional well-being, as well as their social development. In contrast, the Who Am 
I? instrument although appearing to have more clearly established its psychometric properties is 


lacking the physical, emotional and social domains. 


5. In further projects of this nature a number of important considerations appear important from 

results of this study: 

e It would be important to have all stakeholders involved in aspects of the work. 
Administrators would be involved in all in-service activities and discussions so they can 
better support the teachers as well as ensure utilization of the data. 

e Significant in-service activities should be directed towards discussions of how the data may / 
should / could be utilized in making educational, social financial, community and/ or health 
decisions. 

e The data might best be collected in February or March and provided back to schools in May 
of the same year. 

e Monitoring should be in place to ensure data is complete and that all or as many subjects are 
included as possible to circumvent difficulties with large amounts of invalid data. 

e In order to ensure that the impact of the work is on the total environment of the child all 
significant agencies including Alberta Learning, Children’s Services, Community 
Development, Health and Wellness, and the Alberta Children and Youth Initiative 
Recreation, etc. should be involved. This initiative should be a coordinated approach with all 


the stakeholders involved. 
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